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Abstract 

Pennsylvania System of School Assessment (PSSA) data from one rural school system covering four 
groups of children for a consecutive three year period was used to study the impact of transient students entering the 
school system. The analysis compared native children (those on roll since the first year) with transient children 
added to or deleted from the enrollment base of the district during 2006, 2007, 2008 academic years when the PSSA 
exam was administered. Significant differences favoring the achievement of native children were noted over the 
years of the study. A policy recommendation regarding data used for making Adequate Yearly Progress (AYP) is 
also presented in this paper. 
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Introduction 

Believing we can improve schooling with more tests is like believing you can make yourself grow 
by measuring your height. 

Robert Schaeffer 

National Center for Fair and Open Testing 
School districts are struggling to improve academic achievement in this new age of 
accountability. This is a reflection of the fact that our schools are experiencing ever more 
pressure for higher levels of academic achievement as the target date of 2014 for universal 
competence approaches. This mandate for universal proficiency and competence was an original 
component of the A^o Child Left Behind Act (NCLB) (P.L. 107-110, 2002). The central goal of the 
NCLB Act was to close the achievement-gap and provide all children, from all backgrounds with 
the tools needed to succeed in today’s world. For that reason each identified group of children 
must all reach the mandated level of proficiency in mathematics, reading, writing, science, and 
technology each year.^ This benign goal of closing the achievement gap between groups of 
children has taken on a less benignant aspect with the NCLB mandates that Adequate Yearly 
Progress (AYP) be shown toward the 2014 goal (Wright, 2008). 

The mind that is anxious about the future is miserable. 

Santana 

The pressure on schools to meet the AYP requirement each year comes from two 
directions. On one hand, under the NCLB Act every school and school district is required to 
publish an annual report card providing the public with a picture of the achievement pattern on 
state tests. In 2006 almost 25,000 schools (27% of the total number) did not meet the AYP 
standard. Local media are quick to excoriate teachers and administrators for what are typically 
described as “failing” schools. Having local schools meet NCLB mandates can have an impact 
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on an entire community. In the suburbs, test scores influence home value; and, divorce lawyers 
use perceived school-quality in arguing parental custody rights. 

The second source of angst for educators regarding scores is the built-in punishments for 

schools failing to meet the AYP standards. These go far beyond the public humiliation of a 

negative press report, and involve the potential loss of careers. The average score of each 

identified group must show annual progress toward reaching the goal of having 100% of its 

population proficient in the identified areas by the year 2014. Failure of a school to achieve the 

goals will result in the school being listed by the state’s education department as “needing 

improvement.” Three consecutive years on this list may cause parents to consider transferring 

their child to a district that is performing better on the assessment; also, the school must offer 

private tutoring to the low performing children. After four years on the list the school may see 

the faculty replaced and administrators fired or demoted. The state department of education may 

2 

permanently close schools with lengthy histories of not passing the NCLB mandates as well. 

The impact of the mandates for testing has been felt most sharply by curriculum areas not tested. 
The fields of social studies, modem languages, the arts, and health have all languished while 
evermore effort has been put into teaching reading and mathematics. 

Other changes to the curriculum brought about by the high-stakes testing mandate have 
had a negative impact on American education. One problem with the testing program is that the 
measures do not require higher order thinking and problem solving skills (Lane, 2004). Tests that 
are developed and used stress lower order skills and basic knowledge. This makes it easier for 
test developers to design multiple-choice measures, but it shortchanges the curriculum (Posner, 
2004). This type of testing also encourages “drill and fill” test preparation activities so common 
in schools today. It has been argued that the NCLB assessments do not allow the schools to teach 
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children lessons that provide skills correlated to living a purposeful life (Sadker & Zittleman, 
2004). 

The impact on the curriculum is most severe in schools where many children are 
performing at a low level. Even in middle class community schools the rich and varied 
curriculum of the past is now a distant memory. The curriculum has been supplanted by hours of 
drill and repeated practice tests focused on basic skills being tested by the state mandated 
assessment (Dillon, 2006). 

Educators are not the only people who bear the burden of high-stakes assessments. In 
eight states, children cannot get out of third grade until they can “pass the high-stakes test.” In 27 
states, high school seniors cannot graduate until they have passing test scores on a mandated test. 
One result of this excess of measurement is that in 2006-2007 over 100,000 of Elorida’s children 
were not promoted to the fourth grade simply on the basis of a test score. Before we went to war 
against “social promotion” it was educators and parents who made the difficult decision to retain 
a child in grade for another year. In my youth “flunking a grade” was seen as a shameful 
experience. Today, in many poor and minority communities it is all too commonplace. To make 
a bad situation worse, in 2006 over 30,000 children were in the third grade for a third time 
having failed the Elorida exam again (Office of Program Policy Analysis and Government 
Accountability, 2006). These children will be eligible to apply for their driver’s licenses when 
they are in middle school, and run for school board seats when they turn 18 in ninth or tenth 
grade. 

A frequently overlooked reason that schools have average assessment scores that have 
been suppressed is family and student mobility. The impact of moving from residence to 
residence is a significant factor in school performance (Kaase & Dulaney, 2005; NAEP, 2004). 
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National data from the NAEP have documented that children of families that have moved three 
or more times have much lower achievement test scores when compared to children who never 
moved. Socio-economic status may play a role in the achievement impact on children of moving 
families. Wealthy parents move because they can; for them it is strategic and usually well 
planned. This is not always so for poor families who often move because they must. It is likely 
that many foreclosures this past year have contributed to this problem today (Armour, 2008). 
Moves made by families and children of poverty may be far more traumatic than is true of 
children with parents who move to improve their lives. 

A California study found that as many as half of the high school transfers did not involve 
the family actually moving. Factors such as custody and parent rights, violence and gang 
activity, and other problems of socialization are involved in the decision of a teenager to request 
a change of high schools (Rumberger, Larson, Ream, & Palardy. 1999). Another type of moving 
occurs when parents who live in impoverished communities with notoriously poor schools 
smuggle their children into the better suburban schools beyond their communities. This occurs 
frequently in the suburban school systems bordering large cities, and is a clear sign of both the 
devotion of parents to their children and the desperation they feel toward their home school 
systems (Dillon, 2006). 

Findings from California are that 75% of all students make an unscheduled change of 
schools during their 12-year school career. When these transfers and relocations occur there is a 
clear negative impact on the academic outcome for the children involved. Each time a child and 
his family moved to a new neighborhood and school, there was a significant reduction in the 
likelihood that the child would eventually graduate from high school (Rumberger, et ah, 1999). 
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Research Question 

The question of the impact of transient fa mi lies on a rural school system in Pennsylvania 
is the primary research question being addressed by this study. Specifically, the study was 
conducted to determine: What impact the transient students of the district are having on the 
pattern of elementary and middle grade achievement on the mandated statewide assessment? 

Methods 

Sample: The school system used for this study will be referred to as the Granite Rock 
School District."^ Granite Rock enrolls about 7,000 students grades K though 12 in four 
elementary, one intermediate, and two secondary schools. The communities included in the 
district cover almost 120 square miles, and 195 school bus routes transport children each day. 
This district is rural, and the community is dominated by farming, retail, recreation, and light 
manufacturing businesses. The children are primarily Anglo- white with only 7 % of the 
population of students being English Language learners or ethic minorities including African- 
Americans. Data from the state education department indicate the middle schools are scoring 
below the state average in mathematics and above in reading. This district loses very few 
children to transfer each year, and is experiencing a pattern of slow growth. 

Instrumentation: Beginning in 2006, the Pennsylvania Department of Education (PDE) in 
accordance with NCLB mandates expanded its testing program, and began to test reading and 
mathematics in grades 3 through 8 inclusive, as well as in grade 11.^ 

Study Design: The information used in this study was extracted from Granite Rock’s 
PSSA student data files for a 3-year period. Employed in this study were individual results for 
2006, 2007, and 2008 consisting of all students who took the PSSA mathematics and reading 
exams during that time frame. This was a total of 2,293 test score records. Since the goal of this 
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Study was to investigate the results for a consecutive 3-year period, multiple grade levels were 
used as identified in Table 1. 



Table 1 

Grades and Years 



2006 


2007 


2008 


Group 1 3 






Group 2 4 


4 




Group 3 5 


5 


5 


Group 4 6 


6 


6 




7 


7 



Group 1 is the class of 2012 
Group 2 is the class of 2013 
Group 3 is the class of 2014 
Group 4 is the class of 2015 



Thus, there were a total of 12 points for data collection (years by groups) and two tests per year 
(reading and mathematics) collected for the following analyses. 

Analysis 

Data were analyzed by cohort groups. In 2006, the first year for data collection both 
PSSA reading and mathematics tests for children in the third through sixth grades were used. 
Data from these same four cohort groups (the classes of 2012 through 2015) were used to answer 
the research question in this study. The analyses were based on the PSSA standard scores for the 
annually administered test. The Pennsylvania mean standard scores along with the scores from 
both the native population of students (those who were continually enrolled during all three 
years) and for transfer students for each or the three years as well are presented in Tables 2 and 
3. Independent samples “t” tests were used to test for the significance of any difference between 



the native and transfer student scores. The assumption of homogeneity was tested using Levine’s 
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method and all 24 analyses were within the required parameters for the test procedure. There 
were a total of 24 analyses, two tests, over four grade levels, and three years. 

Results 

There is a striking pattern to these findings. With the PSSA reading test 6 of the 12 
analyses were significant. In 5 of the 6 non- significant findings, there was a clear monotonic 
trend for the transient students to have poorer levels of reading achievement compared with the 
native population of students. 

Table 2 



Pennsylvania System of School Assessment - Reading 





Transfer 


Number 


Native 


T 


Sig. Level 


Class Year 2015 


Grade 3 


1243 


31 


1334 


3.09 


.05 


Grade 4 


1281 


49 


1341 


2.10 


.04 


Grade 5 


1212 


38 


1321 


3.42 


.01 


Class Year 2014 


Grade 4 


1255 


24 


1354 


2.49 


.02 


Grade 5 


1272 


20 


1330 


1.20 


n.s. 


Grade 6 


1293 


61 


1376 


3.18 


.01 


Class Year 2013 


Grade 5 


1305 


15 


1337 


0.53 


n.s. 


Grade 6 


1313 


41 


1356 


1.47 


n.s. 


Grade 7 


1320 


57 


1363 


1.54 


n.s. 


Class Year 2012 


Grade 6 


1399 


25 


1358 


0.76 


n.s. 


Grade 7 


1405 


35 


1394 


0.23 


n.s. 


Grade 8 


1369 


64 


1495 


4.12 


.01 



This same pattern was found with the PSSA scores from transient and native students in 
mathematics for this school system. The analysis of these data found that for 6 of the 12 groups, 
the native population did significantly better than the transfer students on the PSSA mathematics 
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exam. The same monotonic trend was found across the non-significant comparisons, with 5 of 6 
indicating better scores for the native population of students. 

Table 3 



Pennsylvania System of School Assessment - Mathematics 





Transfer 


Number 


Native 


T 


Sig. Level 


Class Year 2015 


Grade 3 


1293 


31 


1425 


3.37 


.01 


Grade 4 


1328 


49 


1410 


3.00 


.01 


Grade 5 


1331 


38 


1409 


2.28 


.03 


Class Year 2014 


Grade 4 


1267 


24 


1379 


3.03 


.01 


Grade 5 


1355 


20 


1386 


0.78 


n.s. 


Grade 6 


1353 


61 


1420 


2.28 


.03 


Class Year 2013 


Grade 5 


1269 


15 


1380 


1.90 


n.s. 


Grade 6 


1350 


41 


1390 


1.16 


n.s. 


Grade 7 


1278 


57 


1355 


2.76 


.01 


Class Year 2012 


Grade 6 


1372 


25 


1358 


0.43 


n.s. 


Grade 7 


1415 


35 


1387 


0.72 


n.s. 


Grade 8 


1295 


64 


1363 


1.40 


n.s. 



As a point of comparison, Table 4 provides Pennsylvania’s 2008 statewide average 



scores for reading and mathematics for third through eighth grades. 
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Table 4 

Pennsylvania System of School Assessment, Reading, and Mathematics 
Statewide Mean Scores and Standard Deviation Values 





Reading Mean 


SD 


Math Mean 


SD 


Grade 3 


1340 


139 


1330 


185 


Grade 4 


1370 


225 


1450 


243 


Grade 5 


1330 


222 


1450 


234 


Grade 6 


1360 


221 


1460 


254 


Grade 7 


1390 


235 


1440 


237 


Grade 8 


1480 


273 


1410 


221 



Conclusions 

School systems and their administrators have been placed in an untenable place by the 
high-stakes testing mandate of the No Child Left Behind Act. Research has documented that 
good leadership can have a significant impact on scores achieved by students on the PSSA 
assessments (Cantwell, 2007). Yet, little can be done when less well prepared children transfer 
into a school a few weeks or months before the state mandated tests. Yet, research has also 
demonstrated that each transfer into the classroom reduces teaching efficiency and lowers the 
scores of all students on achievement measures (Rumberger, Larson, Ream, & Palardy, 1999). 

The policy of testing and reporting all students’ scores should be modified. The easiest 
solution is for the states to adopt the policy of requiring all children to have been enrolled by a 
school for a period of 12 calendar months or more before they should be considered part of the 
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district’s database for AYP accountability and school report card results. This may seem to be a 
bit of legerdemain, but it is a practice adopted by the state of Illinois in 2006 (Illinois Association 
of Directors of Title I, 2006). That state changed the rule from, all children on roll since 
September of the academic year must take the test, to all children who have been enrolled for 12 
consecutive months must take the mandated state test, the Illinois Standards Achievement Test 
(IS AT). This small step made it possible for over 50 schools in Illinois, that would have 
otherwise failed, meet AYP requirement that year. 

A second point to be made is that schools should have a plan in place for all students that 
transfer into the district or school system. This should include the use of an achievement 
screening test based on the state’s learning standards. Low scoring students should be provided 
with immediate remediation and learning support. 
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* These groups include: Native Americans, African-Americans, Hispanics, children with special needs, children 
from impoverished homes, English Language Learners, Asian Americans, and Anglo-white children. 

^ This past summer I saw this happen to a local high school. The high school teachers being reassigned were told 
“not to take it personally.” These teachers had given the past ten or twenty years to the struggle to encourage 
learning by high school students. Most of that high school’s students came from impoverished homes where Spanish 
was spoken at home and in the community. On the basis of the annual high-stakes test the school was ordered to 
either close or reorganize. Most of its teachers were summarily reassigned to schools as far as 25 miles away 
(Brannock, 2007). It is no surprise that high-stakes testing in our schools is producing overwhelming pressure on 
everyone. 

^ In 2007 the NCLB Act mandated that science also be tested, but no standards for AYP achievement levels have 
been identified as of the time of this writing. As such, no sanctions will be experienced by schools based on science 
achievement scores in 2008-2009. 

Although the data examined in this study are accurate and true, the Superintendent of Schools has requested that 
the district’s name remain confidential. 

^ Statewide achievement testing started in Pennsylvania in 1963 as a result of the School District Reorganization 
legislation under Act 299. This law brought the number of school districts in the state down from over 2,000 to just 
500 operating systems. The debate throughout the state was contentious focusing on issues such as whether school 
district size is related to education quality. As a result of this legislation, a focus on student personal, social, and 
intellectual growth became priorities. In November of 1965 the State Board of Education adapted 10 goals of quality 
education, reflecting these priorities and requiring the Education Department (at the time the Department of Public 
Instruction) to develop an assessment plan. In the fall of 1973 a timetable was established to assess all students in 
grades 5, 8 and 1 1 using a new measure, the Educational Quality Assessment (EQA) across the Commonwealth 
(Kohr, 2002; Welsh, 1971). 

The next major revision to assessment in Pennsylvania came in 1984 with a newly elected Governor’s 
education reform plan and new assessment system. The assessment was the Testing for Essential Learning and 
Literacy Skills (TELLS), a competency based test for grades 3, 5, and 8. The TELLS assessment was designed to be 
an “early warning system” to identify students with reading and mathematics difficulties. The TELLS assessment 
continued to be the state’s mandated assessment program through 1991. Another new governor was elected in 1990 
bringing with him yet another new plan for educational reform and a new testing program. In 1993 the state’s 
Department of Education introduced the Pennsylvania System of School Assessment (PSSA) for grades 5, 8, and 11. 
The PSSA is a standards-based criterion-referenced assessment used to measure an individual’s attainment of 
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approved academic standards. This testing program matched the new national mandates of the Clinton 
administration under the Improving America’s Schools Act (lASA) (P.L. 103-382, 1994). The exam also determined 
the degree to which school programs enable students to attain proficiency standards (Sinclair & Thacker, 2005). 
Thus, it became a truly high-stakes test. As James Popham (1999) said, “the rules for educators all changed when 
newspapers began to report test data, and even rank schools based upon the achievement outcome of students.” In 
2000 Pennsylvania’s Educational Empowerment Act was passed giving the state’s Education Department the power 
to take-over the management of school systems with consistently low PSSA scores. 





